Locate network observations with a MIKE+ database - #698
Draft
jpalm3r wants to merge 3 commits into
Draft
Conversation
| on_missing: Literal["raise", "skip"] = "raise", | ||
| aux_items: list[int | str] | None = None, | ||
| attrs: dict | None = None, | ||
| ) -> list[NodeObservation]: ... |
| quantity: Quantity | None = None, | ||
| aux_items: list[int | str] | None = None, | ||
| attrs: dict | None = None, | ||
| ) -> list[ReachObservation]: ... |
| on_missing: Literal["raise", "skip"] = "raise", | ||
| aux_items: list[int | str] | None = None, | ||
| attrs: dict | None = None, | ||
| ) -> list[ReachObservation]: ... |
jpalm3r
marked this pull request as draft
August 10, 2026 11:30
jpalm3r
force-pushed
the
read-sqlite
branch
2 times, most recently
from
August 10, 2026 13:50
db99d3c to
29aa481
Compare
The MIKE+ database reader lands next and does not belong in the same file as the Network class. Move network.py to network/__init__.py unchanged so the split that follows is a pure addition. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A network result file identifies nodes and reaches by ID, but observations are usually recorded against real-world station names held in the MIKE+ setup database. Read that sqlite database to resolve a station to the node or reach it sits on, so observations can be placed without hand-mapping every ID. Dataset variables also gain a long_name attribute, so a quantity keeps its label once it reaches xarray. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Building a
NodeObservationtoday means knowing which node each sensor sits on, which usually ends up as a hand-written SQL query in a notebook. A MIKE+ project already records that:m_Measurementsays which file and item each measured timeseries lives in,m_Stationsays where in the network it belongs.This adds a
dbargument toNodeObservation.from_multipleandReachObservation.from_multiplethat does the lookup.The mapping is keyed by item, not by location
One dfs0 item maps to exactly one location, but a location can carry several items — two pressure transmitters either side of a check valve, three flow meters on one pipe. So the lookup returns one row per item and one observation is created per row. The existing
nodes={location: item}dict cannot express this, since a location can only appear once as a key; it still works, and its docstring now says what it cannot do.Which class you get is decided by the database
m_Station.locationtypesays whether a station sits on a node or a link, and it is load-bearing: somelocationidvalues match both a node alias and a reach id, so trying node-first-then-reach would silently resolve the wrong one. Junctions and tanks becomeNodeObservation, links becomeReachObservation, and a link carrying a chainage becomes aNodeObservationat that breakpoint. Asking one class for a quantity the database places on the other raises and names the class you want.Contained behind a fixed contract
network/_mikeplus.pyreturns a fixed set of columns —item_name,name,location,kind,quantity. Table names, the join, thelocationtypecodes and theresitemnameencoding stay inside that module, so a change to the database layout is a change to one file.obs.pynever sees a MIKE+ concept.network.pybecomesnetwork/__init__.pyto make room for it. Pure move — every import in the repo isfrom modelskill.network import ...and is unaffected.Other changes
ReachObservation.from_multiple, mirroring theNodeObservationclassmethod, which did not exist before.from_multiple(nodes=...)keys now accept aliases and(reach_id, distance)breakpoints.NodeObservation.atalready took all three; onlyfrom_multipleadvertised integers.Network.to_dataset()keyed its data variables by quantity name but never wrote it to the DataArray attrs, soNetworkModelResultreportedQuantity.undefined()and skill tables labelled the model column with the observation's quantity. res1d and EPANET files carry a name without a unit, andQuantity.from_cf_attrsneeds both, so it falls back to the name alone.Notes for review
Undefinedfor every item, so the database is the only reliable source for the name; the unit still comes from the data.on_missing="skip"builds from the rest.match()still does not compare an observation's quantity with the model's; that is a wider change, tracked in Quantity.is_compatible reports undefined quantities as incompatible #697.Verified end to end against a real EPANET model and its MIKE+ database: 30 observations, 30 comparers.